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Navigation — knowing where one is and finding a safe route— is a fundamental 
aspect of all exploration. In unfamiliar terrain, one may use maps and 
instruments such as a compass or binoculars to assist, and people often 
collaborate in finding their way. This paper analyzes a group of people driving a 
humvee from a base camp to the north coast of Devon Island in the High 
Canadian Arctic. A complete audio recording and video during most stops 
allows a quantitative and semantic analysis of the conversations when the team 
stopped to take bearings and replan a route. Over a period of 2 hours, the 
humvee stopped 20 times, with an average duration of 3.15 min/pause and 3.85 
min moving forward. The team failed to reach its goal due to difficult terrain 
causing mechanical problems. The analysis attempts to explain these facts by 
considering a variety of complicating factors, especially the navigation problem 
of relating maps and the world to locate the humvee and to plan a route. The 
analysis reveals patterns in topic structure and turn-taking, supporting the view 
that the collaboration was efficient, but the tools and information were 
inadequate for the task. This work is relevant for planning and training for 
planetary surface missions, as well as developing computer systems that could 
aid navigation. 


INTRODUCTION: PRACTICAL AND SCIENTIFIC OBJECTIVES 

During Apollo lunar traverses the astronauts relied on CapCom, a Mission Control 
point of contact in Houston, to help them navigate and use their time wisely. Until roads 
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are built and signs deployed, future lunar traverses will pose the same navigation 
difficulties encountered in Apollo, especially when exploring new areas. Even robotic 
teleoperations from earth will require teams to relate imagery to their maps and route 
plans. Consequently, it will be useful to better understand how people navigate using 
maps and what problems they encounter in coordinating what they see, interpret, and say 
when driving a vehicle in hazardous terrain. 

In particular, Apollo’s lunar traverses illustrate that detours are often required when 
the terrain is unfamiliar and irregular, and these detours necessitate repeatedly re- 
establishing one’s location and replanning a route, This problem occurs despite learning 
to recognize and name landmarks (such as craters) from photographs, as exemplified 
while driving the rover in Apollo 17 1 : 

146:22:32 Gene Ceman: Look at that. Right on the southeastern ... 

146:22:34 Jack Schmitt; Now, wait a minute. 

146:22:35 Cernan: ....(Correcting himself) southwestern rim. 

146:22:36 Schmitt: Yeah, yeah. 

146:22:37 Ceman: (Perhaps looking at cuff checklist #22) Yeah, because Horatio's 
got to be on our right. Well, wait a minute, doggone it. 

146:22:41 Schmitt: It's not Horatio, is it? 

146:22:43 Ceman: Well, we're at 094, 1.7. 

146:22:46 Bob Parker (CapCom): Stand by. (Pause) 

146:22:52 Schmitt: No, I think that's Camelot. Horatio didn't... 

146:22:53 Ceman: That's too.. .That's too... 

146:22:54 Schmitt: ...have blocks that far up the rim* 

146:22:55 Cernan: ...Let me.. .Yeah, let me look in the bottom. I'll tell you. I 
remember. 

146:22:58 Schmitt: Yeah. 

146:23:00 Parker: That kind of sounds like Camelot to us. ... 

Even using today’s technologies like a global positioning system (GPS) that might 
pinpoint location automatically, a route suitable for the mobility of the transportation 
vehicles and robots must still be found. Routes must adapt to local conditions (e.g., cliffs 
and loose rocks) and therefore looking ahead during travel is required. Jumbled lunar 
terrain required many detours 2 : 

142:26:25 Ceman: We're doing a little zig-zag navigation. I literally came up a 
slope at about a heading of 240 (WSW). We couldn't get through the actual turn to the 
south because there is a big crater right at the foot of it. So we're just making our way 
through some relatively local undulating slopes that get pretty steep, but it seems to be no 
problem. 

In such circumstances, explorers need to articulate their location hypotheses and 
route plans to each other: 

142:31:58 Ceman: Jack, I'm going to head right along this ridge because I think 
that's the depression we were talking about. 

142:32:01 Schmitt: Yep, that's Nansen down there. 
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142:32:02 Ceman: Where are you looking? Right there? 

142:32:04 Schmitt: I think, right below... 

142:32:06 Ceman: I think you're right. I think that's it. Let me get over here, and 
then I'll head a little bit to the south, 

142:32:12 Schmitt; Yeah, we're a little more west, I think, than we intended to 

be.... 

142:33:29 Ceman: Bob, my best guess - let's see, 077, 7.7, 6.6 - is that we're 
coming up on the northern side of Nansen. 

An external assistant, such as CapC-om, using perhaps different maps and lacking 
the astronauts’ point of view, as well as not seeing their gestures, must sometimes 
struggle to follow along 3 : 

144:22:22 Schmitt: I've sort of lost track (of the two bright craters).., 

144:22:23 Ceman: We're about there (pointing at the map). 

144:22:24 Parker: I think we expected you guys to be a little bit farther north... 

144:22:26 Ceman: I think we want to be more to the left. 

144:22:26 Parker: ...We were guessing a heading (means "reading") of 080 for the 
bearing (to the LM) which really kind of says you were going a bit farther north than this. 

[Parker -is confused 080 was the bearing at the cancelled LRV stop. Houston 
estimates that Station 3 will be at 089/6. 1. Gene (Ceman) is currently at 087/5. P, a short 
distance southeast of where they want to be.] 

Such interactions raise questions about automating CapCom’s navigational role. If 
Bob Parker became confused when relating the current route to the plan and adjusting the 
plan, how could a computer program do better? Smaller robots might find a way 
automatically or through teleoperation from a base camp, but such routes might prove 
impassable for larger vehicles. Whether we are considering people exploring or robotic 
scouts or some combination, we need to learn much more about navigation in unfamiliar 
terrain so we can build the right tools and adopt practical operations procedures. For 
example, is a crew of two likely to be so overburdened that an external CapCom is 
necessary? Could a computer assistant automate some of these navigation tasks or at least 
help CapCom follow along? 

Analyzing the historical record of Apollo traverses is an excellent way to 
understand the navigation problems we will encounter on the moon and Mars. 4 Studying 
exploration in unfamiliar terrain on earth, particularly in lunar and mars analog 
topographies, provides another opportunity, which is the approach of this paper. We 
study people handling serious location and route problems, focusing on the strengths and 
limitations of their decision making and communication practices. 

The present study involves two scientists driving a humvee accompanied by two 
“scouts” on all-terrain vehicles (ATVs), carrying out an operational test in unfamiliar 
terrain to the north coast of Devon Island in the Canadian High Arctic. This group 


TT This italicized note was inserted by Eric M. Jones, the editor of the ALS J. 
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constitutes a small “away team,” on a one night trip from base camp. To replicate a 
protocol used on an earlier humvee trip, in which the scouts are simulating robots, GPS is 
used only to establish waypoints and record progress, not to navigate. The experience of 
this group reveals that GPS would have saved a great deal of time in locating the humvee, 
but would not have been sufficient for finding a safe and efficient route. 


This- paper approaches the humvee trip from two' perspectives: asran analog' ffifsston 
report with practical implications for planetary exploration and as basic research in 
cognitive science. The analysis presented here will straddle these practical and scientific 
perspectives, on the one hand seeking to explain the outcome of the operational test and 
lessons learned, and on the other hand, to explore the data in an open way to learn about 
decision making dialogues during a navigation task. 

The scientific study of navigation involves inves tiga tion in natural work settings, 
recording teamwork, and relating diverse perceptual cues, gestures, and artifacts (maps 
and instruments). For these reasons, the problem' of group navigation was mostly ignored 
in psychology until researchers took seriously that cognition is situated (located 
physically and socially conceived), distributed (among people, computational devices, 
and modalities of representation such as gesture and speech), and interactive (involving 
feedback on different levels with other people and machines). 5 Chase and Chi 6 pioneered 
the study of spatial reasoning in navigation by relating routes to the hand-drawn maps of 
taxi drivers. Hutchins work 7,8 is probably the most well-known synthesis of situated 
cognition issues in a navigation setting, particularly his studies of an airplane cockpit. 

' As the Apollo transcripts make obvious, the teamwork of navigation can involve a 
complex dance between pilot and co-pilot, involving looking, pointing, describing, and 
questioning. Such exchanges, called turn-taking , have been formalized in the field of 
social linguistics since the 1970s. 9 The data collected in the humvee affords studying how 
the sequence of topics relates to different speakers. The speakers have different roles in 
driving and navigating, and this is manifest by how they focus on the world and the map, 
as they coordinate locating where they are and finding a route. 

The present investigation is not an experiment with pre-defmed variables and 
hypotheses. Rather, as much as the trip for the geologist and biologist involved exploring 
new terrain, the trip was also exploratory for the cognitive scientist. One might call this a 
“natural experiment,” in the sense that the scouting protocol involved well-defined goals, 
players, communications, and limited interaction with the outside world. On the other 
hand, the trip was not designed to produce useful information about decision making, 
cognitive processes in using aerial photographs, or teamwork. Rather, such issues and 
their relevance were discovered along the way, in the manner of geologists and biologists 
discovering in the field materials and patterns relevant to their interests. Hence, the 
analysis of the navigation dialogues includes a strong classificatory aspect of sorting 
through data to find statistically significant patterns. 
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The humv.ee pilot and navigator thought they would arrive at the north coast by 
midnight; their appraisal of the task was incorrect. This becomes the starting point of our 
study of their decision making and communication practices. Using the audio and 
photographic record, we work backwards to analyze the outcome of traverse and how this 
relates to individual and group decisions. Thus a central research question. of this paper is, 
what can we learn about the difficulties of the navigation task from the dialogues in the 
humvee? ■ 

In our analysis, we find that the excursion suffered from lack of information (both 
locally and globally in the route). Perhaps intentionally, to carry out the experimental 
route-finding protocol, they did not reflect on trends and the impact of recurrent problems 
on the overall goal to adopt a different strategy, From these difficulties we can posit 
planning, route-mapping tools, and alternative protocols (e.g., different scouting 
methods) that might have ensured success in reaching the coast. 

Regarding our practical concern with planetary ' exploration, understanding 
navigation is directly relevant to an ongoing research project, called Mobile Agents. 
Using the rubric of “automating CapCom” 10 ’ 11 , we are developing computer agents that 
astronauts can voice command during a extravehicular activity (EVA) to control devices 
(e.g., cameras, robots), store and access EVA data (e.g., name places and comment on 
photographs), and receive location, procedural, and schedule advice. During such speech 
interactions, the computer system must “take turns” in a way that follows conventional 
patterns, such as not interrupting people when they are talking to each other, respecting 
authority 12 , confirming commands and signaling problems, and making contextual 
inferences. 13 Related robotic research has considered how to develop “conversational 
agents” with informative gaze behavior (e.g., looking toward the listener). 14 Analysis of 
the pilot-navigator discourse may be relevant for constructing a simulation 13 of the 
navigation practice of the team, which could in turn be used to specify and test an agent 
that provides navigation assistance. 

Subsequent sections of this paper describe the experimental setting and recording 
methods, hypotheses about the outcome of the operational test, an overview of timing and 
topics salient in the data, and a work practice analysis that characterizes the navigation 
discourse as it relates to route decision making. The paper concludes with a summary of 
the results and implications for future work. 

EXPERIMENTAL SETTING: A HUMVEE OPERATIONAL TEST 

This navigation experiment was part of the annual Haughton-Mars Project 
expedition. 16 The humvee pilot, identified in this paper as B, had previously led a group a 
week earlier to cross Devon Island using aerial maps, bringing the humvee about 74 km 
(40 nautical miles) from the west coast to base camp. The intention now was to make an 
overnight trip to the north coast of Devon Island, about 20 km away, to reach the gullies 
(generally ice-filled cliff ravines) investigated by helicopter in 2001. This traverse was 
part of a plan to conduct tests of the vehicle over several field seasons, to learn its 
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operating characteristics, issues in navigation, habitability of the vehicle, and so forth. By 
yielding a great deal of such information, the operational test reported here was a success. 

The group held a planning meeting on Tuesday July 22, 2003 to configure the 
navigation process. It was decided to replicate the method used in navigating from the 
west coast, with a person on an all terrain vehicle (ATV) scouting ahead of the humvee. 
The' pilot (B)"Snd co-pilot (tj)' would ride' up 'front; die observer (the first author) would 
sit between with cameras and audio-recorder; two ATV riders would scout, one nearby to 
show firmness of the ground and one further to anticipate the route. 

The chosen navigation method assumes that GPS is not available; instead location 
is determined by relating geographic features (hills, ravines, and permanent snow 
patches) to black and white aerial photographs (Figures 1 and 2) that were mostly taken 
in 1958-59 from an altitude- of about 9 km (30,000 ft). The spatial resolution (smallest 
identifiable feature) in these photographs is approximately 4 m (i.e., we can spot boulders 
4 m across, but none of the smaller ones unless they stand out against their background 
by albedo, in which ease they could be as small as 2 m across). A topographic map was 
available, but was used primarily to plan the initial route on the aerial photos. 

The navigation process involved: 1) determining the humvee’ s location with respect 
to aerial photographs (referred to as maps; mostly managed by G), 2) learning about the 
local area (from the ATVs driving in front of the humvee, one nearby to indicate wet 
areas, the other more distant to scout routes), 3) planning a route in the world by 
integrating photo and local information with what is visible from the humvee (B’s 
decision based on G’s advice), and 4) navigating local obstacles with respect to the 
chosen plan (B’s task as driver). 


The group of five departed late afternoon July 24. The discourse data and analysis 
we present comes from the initial 2 hours of route finding; the outcome analysis 
considers as well the interactions with the scouts. 

The observer sat between B and G in a makeshift seat. The entire trip was audio 
recorded, using an mini-disc (MD) digital recorder. The microphone dangled between B 
and G (subject to some wind noise when windows were open). The initial route planning 
and most conversations were video-recorded with a hand-held mini-digital video (DV) 
recorder using 90 minute tapes. Both the video and sound quality are excellent. A digital 
wide-angle camera was used to record the general setup (Figure 1). Photographs and 
video still frames were recorded periodically (Figure 2). 

PROBLEM: WHY DIDN’T THE GROUP REACH THE COAST? 

As mentioned, the objective of the operational test was to learn about using the 
humvee for exploration. For this purpose, the group chose the goal of reaching the north 
coast of Devon Island. 
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Figure 1. Humvee^cockpit-with-B-Ou left,_G on nglit,auapS-and-aenaI photos shared 
in the middle. (Photo: W. Clancey/NASA Haughton Mars Project) 


Figure 2. B and G determine where they are and where they want to go. 
(Photo: W. Clancey/NASA Haughton Mars Project) 

Departure was delayed until after 5 PM, with midnight as a hopeful target for 
arrival. With the Arctic summer sun, it would remain light through the night and the 
group was self-sufficient for food and shelter, so stopping anywhere would be acceptable. 
The weather was mediocre, cloudy with cold winds and periodic light showers. 



Two factors are most salient in the outcome of the traverse: First, an inappropriate 
route was chosen (after considerable deliberation), which caused a mechanical failure that 
necessitated spending the night about 6 km (linear distance) from base camp. Second, 
insufficient time was available for the trip, necessitating the group to turn back at noon 
the second day, before reaching the objective. 
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In this section we consider decisions made during preparation that affected the 
traverse, recurrent issues that arose during the trip, and hypotheses explaining the 
outcome, which are examined subsequently in the data analysis. 

Preparation decisions 

During plannin^idiscnssrons' at' 'severahmeetings prior to the humvee traverse, the 
group carefully articulated their individual objectives and constraints. These included: 

• research (gullies, general exploration, experimental scouting protocol) 

* safety (on traverse and affect of group’s absence on rest of the camp) 

8 logistics (e,g., number of people who can sleep inside humvee) 

8 comfort (e.g., don’t ride ATV in bad weather such a long distance) 

* maintenance (of humvee and AT Vs) 

8 other camp activities (e.g., phone calls, other camp commitments, media 
visits, ongoing communications experiments) 

• documentation (the scouts included a Cinematographer and writer) 

The consensus was that all objectives had to be accommodated, perhaps through some 
adjustment of external constraints (e.g., rescheduling a media event). The weather 
introduced some uncertainty that probably contributed to the lack of a committed 
departure time. 


Recurrent issues during trip 

During the trip, the group was strongly focused on the task at hand. The recurrent 
issues included: 

• Where are we? 

• What route should we follow? 

» Are obstacles ahead on the present path (e.g., boulders, mud, steep hills)? 

• Where are the scouts and what are they doing? 

• How could the scouts be helping better? 


These issues are represented formally in the analysis of the navigation dialogues. 
Hypotheses: Possible causes for not reaching the coast 

After considering the data and experience in some detail, we suggest the 

following possible causes influencing the outcome of the trip. At this point, these are not 
conclusions, but hypotheses that analysis may support or refute. Three factors are posited 
at the top level: 1) The terrain made the trip impossible using the humvee with tracks; 2) 
The time available was insufficient, given the terrain and the navigation method; and 3) 
The navigation method was inadequate, given the terrain and travel time available. The 
time and method factors in turn have underlying, root causes to be considered. 
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• Terrain inappropriate for track humvee: Mechanical failure was caused by 
small rocks becoming caught in the humvee’ s tracks. These effects were 
possibly under estimated because earlier in the month the terrain was wetter 
and mud was the primary concern. This hypothesis claims that regardless of 
the location and route-finding practice used by the crew or time available, 
using tracks the humvee would not be able to make this trip safely. 

• Insufficient time available for the tiip: Departing at 5 PM allowed seven 
hours to travel 20 km. The logged location after about one hour was 5 km 
(linear distance) from camp, which is consistent with the plan. However, 
including the additional distance traveled until him around on the second day, 
the group covered about 10 km (linear) in five hours (2 km/hr). Consequently, 
to reach the coast (20 km) might have required 10 hours. This hypothesis 
claims that given the terrain and navigation method, the trip could not be done 
in the available time. The root causes include: 

o Miscalculation: The 74 km trip from the west coast two weeks earlier 
required several days (with an approximate linear rate of 4.5 km/hr). An 
assumption might have been made that the northern route was better, as 
the seasonal mud had dried and there were no rivers to cross. Thus, even 
though the same navigation protocol would be used, it was tacitly assumed 
that travel would be at least as fast on this trip. 

o Higher priority goals: Other goals caused the group to leave late in the 
day and required them to return by early afternoon the next day. These 
concerns included lack of time for media visitors to meet with the humvee 
group participants, ongoing communications tests that required the 
humvee to be in camp, and an important incoming phone call (that 
necessitated being at base camp). Furthermore, spare parts for the humvee 
arrived unexpectedly, and it was decided to use them for maintenance 
prior to departure. 

• Navigation method was inadequate: Th is hypothesis claims that assuming 
that the terrain afforded a safe route for the humvee using- tracks, the available 
time would have been sufficient if the location and route-finding method were 
improved. This hypothesis has several interacting root causes: 


o Aerial photographs were insufficient for navigation: The photographs 
(taken at 30,000 feet) are difficult to relate to the visible terrain. Thus, the 
humvee was moving faster than the navigator could track their location, 
necessitating frequent stops. Also, the task might have been sufficiently 
difficult that it required the pilot’s assistance. The navigator may have 
been preoccupied by locating the humvee, unable to devote sufficient time 
to route planning. 
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o Route needed to be planned more globally: Local obstacles were leading 
to a chain of long detours, which introduced uncertainty in the overall 
route, making it difficult for the distant scout to stay on the humvee’ s path. 

o Scouts didn’t provide needed local obstacle and route information: For 
example, they were not properly placed, not communicating/coordinating 
withmaavigatar appropriately; • misinterpreting terrain affects on humvee; 
and were impaired by weather. 

How should the analysis proceed, given the interaction of these factors? One 
could speculate that if the humvee were using wheels instead of tracks, then the time 
available and navigation method might have been sufficient. But regarding the terrain, 
one could also argue that the west coast traverse demonstrated that the humvee can get 
through Devon Island’s terrain, so it remains of interest to know how to navigate better 
with tracks. Although we do not know what difficulties lay between the stopping point 
and the coast, the recorded conversations suggest that an alternative route could have 
been found to circumvent the terrain that caused mechanical failure. 

Regarding the time available, the expediencies of the short summer season made 
attempting the operational test more advantageous than canceling, even if the time was 
short. One could argue that 4-5 km/hr (linear) was a good estimate, and if a better route 
had been found, would have been sufficient. 


Thus, although it is quite possible that wheels or more time would have enabled 
the group to reach the coast, our analysis of the data will be most productive if we focus . 
on the navigation method. To this end, subsequent sections provide an overview of the 
data, followed by analysis of the navigation method and conclusions. 


DATA OVERVIEW: TIMING AND TOPICS 

The audio and video recordings made during the traverse provide the main source 
of information for evaluating hypotheses about the use of time and how decisions were 
made. As stated in the introduction, a related scientific motivation in analyzing the 
dialogue is to learn what aspects of the task are difficult, thus causing delays or 
contributing to non-optimal decisions. This section describes the basic analysis method 
and the broad facts and patterns it reveals. 

• The-audi0-reeordings-0fAFuIy-24-fr©H^474OO-t9— 20400-were-tr-anscribed-using— a — 

spreadsheet. Columns record the start time of a passage, the status of the vehicle 
(moving, stopped), the duration of the passage (computed using Visual Basic macros), 
and the transcribed text. The initial analysis determined that between 17:20 (when the 
group departed base camp) and 19:40 (when the mechanical problem developed), the, 
humvee was stopped 20 times for an average duration of just over 3 minutes (Table 1). 
Perhaps surprisingly, during nearly half of traverse the humvee was stopped, and the 
average time moving between stops was under 4 minutes. Subsequent analysis is directed 
at understanding this pattern and what was happening during the pauses. 
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Table 1: Comparison of movement and paused durations on leaving camp 


EVENT 

COUNT 

TOTAL TIME 

AVERAGE 



(h:mm) 

DURATION 




(minutes) 

Pause 

20 

1:03 

3.15 

Moving"' forward" 

20- ; 

l':T7 ' 

3.85" 


To carry the analysis forward, three additional columns were added to the 
spreadsheet: Individual rows for each statement “topic” (a categorization of what is being 
said), the person introducing the topic (the pilot, designated as B, or the navigator, G), 
and the topic category. Four general topics were identified: 

• WORLD=>MAP : Identify current location or salient landscape feature on 
aerial photo 

• MAP=> WORLD : Find salient aerial photo feature in landscape 

• ROUTE ON MAP: Determine general route in the aerial photo 

• ROUTE IN WORLD: Determine target heading and route in the landscape 


B and G engage in other shared activities, indicated by the topics WORLD and 
PROCEDURAL in the transcripts (note that B’s remarks are always in parentheses): 

• WORLD: Establishing a shared feature in the world, e.g., G: “Do you see 
where that first snow bank is <pointing out right window> (B: which one?) 
« open window» first one to the left <pointing> (B: right, the big thick 
one?) yeah, (B: okay)” 

• PROCEDURAL: Meta-topics, e.g., indicating an interest in stopping, 
positioning the scouts for providing useful information, relating the planned 
path to the actual route. 


Note that the dialogue has been segmented into groups of utterances (sentences) that 
are about a single topic, as defined above, constituting the unit of analysis. Thus, the 
analysis of turn-taking concerns topic exchange, with the person who introduces a topic 
called the actor . Each topic is sometimes called a “statement” here, though it often 
includes several utterances (ranging from sentences to single words, most commonly, 
“yeah”). Within each topic segment in the dialogue one can find a form of turn-taking 
from one utterance to the next, called speaker exchange, which is not the concern of this 
analysis. Although the original work on turn-taking 9 examines patterns independent of 
the topic, our interest is precisely the opposite, primarily focusing on topic relations and 
secondarily on correlations with the speaker. In particular, we want to first understand the 
patterns of topic relationships (i.e., recurrent sequences) and then how the speakers take 
turns within these sequences. 

On reviewing the audio transcripts, it was determined that the numerous references 
to “this” ' and “that” could not be understood without examining the video recording. 
Thus, the entire transcript was reviewed and further annotated to indicate where B and G 
were pointing. This analysis revealed that the dialogue between B and G concerning the 
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aerial maps and route, which was the purpose of nine of the 19 stops, could be classified 
simply; the detailed analysis below focuses on their interaction during these nine stops. 
The remaining ten stops included: one near the beginning when a camera man swapped 
places with G.; three pauses to log information; one call from base camp; and one rest 
break (7 min.). In addition there are four interactions with the scouts, including 18 min. 
of the final 24 minutes. 


Figure 3 shows the entire first day traverse graphically. Notice that the stops 
become much longer in the last third. This is when the difficult terrain was encountered, 
which we will examine subsequently. Other patterns to notice are the relatively long 
movement periods in the first two-thirds of the traverse, as well as nine stops of one or 
two minutes. 
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Figure 3. Duration of humvee stops during the traverse (black areas represent when 
humvee is not moving; narrow lines are 1 minute or less; stops separated by motion 
for a few seconds are represented as one stop) 

Table 2 summarizes the nine stops (29 min.) when B and G discuss their location 
and route. Another location discussion that occurred when the humvee was not stopped 
(17:56) is included (however, as B is driving during the 17:56 discussion, primarily G is 
speaking and B does not make references to the map). Note that the noise in the humvee 
precluded normal conversation and would explain the need to stop if joint decision 
making were preferred or required by the task. 


B and G each initiated five of the ten iocation/route discusstomswDuring-these- 
discussions B (pilot) starts by referring to the route in the world (ROUTE IN WORLD, 3 
times) or the map location of the humvee (W ORLD=>MAP, twice). G (navigator) 
mentions the relation of the map to the world (once), the route in the world (twice) and 
map location (twice). Neither starts by referring to the route on the map (ROUTE ON 
MAP) or how a feature on the map corresponds to the world (MAP=> WORLD). 
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The first stop after leaving camp (17:50) involved a representational coordination 
that never occurred again, namely identifying in the aerial photos features and waypoints 
that were established when planning the route using the topographic map. This passage 
serves as a good introduction to the transcript analysis. Referring to Table 3, note for 
example that the statement “we’re down here” while pointing on the map is classified as 
“World=>Map.” The place labeled Marine Peak on the topographic map is related to the 
aer-iafmapMhen-frem-the aeHalmapi-s4oGated4n-the-world-(f‘behincLus’-’),.T i 'o-unof theJ_l 
identified topics in this passage are “Topo=>Map,” but this topic does not occur during 
any subsequent conversation oil that day. Indeed, we can identify four distinct phases 
during the traverse: 

1. 17:00-1 7:50 — Materials organization and route planning and navigation 
procedures. 

2. 1 7:50- 1 7:54 — Orientation of maps and photos to each other and the world. 

3. 17:54-19:1 1- — Navigating to the waypoints through the terrain. 

4. 19:11-1 9:44 — Coordinating local route with scouts. 

The quantitative analysis of navigation discourse considers only phase three. 


Table 2. Summary of passages when B and G discuss the navigation, indicating start 
time, duration, who initiated the discussion, the main topic in parentheses, and an 
illustrative statement. Except for 17:56, B stopped the humvee at each point. 


TIME 

DUR 

NAVIGATION MOTIVE 

17:50 

1 

G (Map=> World relation) - , “...get a sense of the topography of these 
things.” (omitted from quantitative analysis) 

17:54 

4 

B (Route in World): “Is there another entrance into that valley?” ■ 

17:56 

4 

B (route in world): “Which side do you recommend, G, that we go 
down this side, . . ?” (Not stopped, but included in analysis) 

18:00 i 

<1 "i 

G (Route in World); “...go to the right there.. ,” 

18:03 

3 

G (location: Woiid=>Map): “Just check where we are.” 

18:23 

1 

B (location: World=>Map): “We’re down here.” 

18:28 

1 

G (location: World=>Map): “I think this is this down here.” 

18:39 

1 

B (location: World=>Map): “You need to know exactly where we are” 

1.8:53 

2 

G (location: World->Map): “We should stop for a minute and figure 
out where we are” . 

19:00 

11 

B (location: Map=> World): “Does this make sense? Is that the valley 
down there?” 


Table 4 provides a complete transcript of one of the longer stops, illustrating a 
typical interaction and how the conversation is segmented. Notice that B and G take 
turns speaking (speaker exchange; i.e., B row is followed by a G row and vice versa), but 
sometimes will speak about multiple topics during a turn (topic exchange; i.e., a G row 
follows a G row or a B row follows a B row). Procedural segments occur usually at the 
start or end of a discussion and are skipped over for purposes of analyzing the navigation 
process. Notice that without the gesture information it is impossible to know what B and 
G are referring to, as the words “this,” “that,” “here,” and “there” are all used to refer to 
both features in the world and on the map. 
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Table 3. A unique discussion relating topographic map to the aerial photos and 
world features (17:50, duration 1 min). B’s statements are in parentheses. Gestures 
are indicated by bracketed references. Other statements are uttered by G. MAP 
refers to an aerial photo; TOPO refers to a topographic map. Bold font indicates 
the main topic of the segment of the dialogue appearing in a given row, classified by 


STATEMENTS- - 

-ACTOR- 

OPERATION 

(Sorry say again)... Get a sense of topography on these 
things, This is Marine Peak <pointing on map with 
pencil> (right) 

G 

TOPO=>MAP 

and we're just heading...! think down here <pointing 
out front window, then onto map>... maybe not.. 

G 

ROUTE ON MAP 

Marine Peak is behind us... 

G 

MAP=> WORLD 

(right, so these are the two... < pointing on map>) 

B 

TOPO- > MAP 

(that's the Anderson Pass thing < pointing on map>) 
yeah 

B 

TOPO=>MAP 

(and we just came down on the back slope,) 

B 

ROUTE ON MAP 

(so what' we're seeing is this <pointing on map>...) I 
think and... 

B 

WORLD=>MAP 

(and so the yellow brick valley actually was. before 
Marine Peak < pointing on map>, quite a bit before. Oh 
wait...) 

B 

TOPO=>MAP 

(so, yeah because see, we're down by here now 
<pointing on map>) right 

B 

WORLD = >MAP 

(so we need to head, that way < gesturing direction 
on map with pencil>) that way 

B 

ROUTE ON MAP 

(so why don't we go back up cpointing out front 
window with left hand>, over the ridge?) Just hang a 
right... 

B 

ROUTE IN ' 
WORLD 


Table 4. Navigation discussion between B and G during pause 18:03-18:06 (B’s 
utterances are in parentheses); <gestures are indicated by brackets>. 


Just hang on a second there, B... just check where we are, I 
think, this hill is this one over here? 

G 

WORLD -> MAP 

(yeah, this hill here is that dome) 

B 

MAP -> WORLD 

so we could just cut across I suppose (yes) if we carry on 
this direction, we could have them scout to the right here 

G 

ROUTE ON MAP 

< pointing on map> 



over there <gesturing outside> 

G 

MAP -> WORLD 

(But, we're supposed to make it to our waypoints,) okay 
(but at the same time...<G passes pencil to B, who takes 
map and looks at it, and proposes a way to modify 
waypoints>) 


PROCEDURAL 
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(So we're right here right now, correct.) We're a little bit 
further on (really?) Yeah, I think we passed, that's the 
beginning of the valley, or at least up there, (yeah, okay) 

B 

WORLD=>MAP 

And we're looking out that way cturning to front 
window> 

G 

MAP=>WORLD 

(we got that thing there clooking on map>,) 

B 

WORLD=>MAP 

Cthis other hill <cloGks Gut Jeft window> to the rJght) 

-B- 

MAP=->W0RLD 

this material on the right here < looking out front window> 
looks a lot better ... (yeah <holding point on map with 
pencil>) and I think if we just carry on straight ahead 
(it's still not that way we want to go, it's that way < gesture 
outside?>) Well, it depends how far you... 

G 

ROUTE IN WORLD 

(because the map is oriented like this, G.. so we're here 
<pointing on map>) right 

B 

WORLD- > MAP 

(you see, there's the big dome there < referring to 
outside>, right?) yes (and there's a lower dome to 
right of it?) correct. 

B 

WORLD 

(that thing <outside> is that <on map>, right?) yeah, 
yup. 

B 

WORLD = >MAP 

Straight ahead? Well... drop down into that valley and... 
<gesturing out front window > (so we need to go around...) 

G 

ROUTE IN WORLD 

(vou see that valley <looking outside>?) Yeah (that we 
see disappearing in the distance,) 

B 

WORLD 

(that's what we want to follow, right?) 

B 

ROUTE IN WORLD 

(isn't it that? <pointing on map>) hang on., (it's this, it's 
this here < pointing on map>) < looking out and back to 
map> yes, it's this bit here <pointing with pencil> 

B 

WORLD = >MAP 

(yes, so that's the direction we want to go, right? 
<gesturing on map>) yup (to get up there) 

B 

ROUTE ON MAP 

yes, I agree, so just straight down this hill < gesturing 
out window with pencil> (Over there...) towards that valley 
<both looking out window> 

G 

ROUTE IN WORLD 

(and could that possibly be even our high point over 
there, in the great distance?) no, I think it's < referring to 
visible hi!l> this closer one clooks on map> ... (no, < points 
on map> the great distance.... you see over there 
< looking out>, the great distance? Is that our 1500 or 
1150? < looking on map> which is where we're headed <G 
takes out another aerial map and aligns it on riqht with first 


t * / /"vrO-! r-v_ k/j * n 

map> Could be couldn't it?) <points on new map and looks 
out windows Which one are you looking at? cboth look 
outside> (the farthest point, the highest point you 
see on the horizon <pointing with left finger>) yup 

< looking at map> (that could be) that could well be (our 
1155) so if we're looking over towards this way, it'd be over 
to the right... might be (I think that makes. sense, right?) 
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Well, we're almost directly.. If we're looking down there 
Cgesturing out window>, you're looking down this 
valley <gesturing on map> aren't you (right) this con 
map> seems further to the right (well, we could be looking 
down this con map>) 

G 

■ 

WORLD = >MAP 

(Anyway, we're headed down that way, is that 
correct?) yeah, we're headed down that way cgesturinq 
outside>. 

. B. 

. ROUTE IN .W ORLD 


Figure 4 shows how pointing on. the aerial photo typically occurs, using a finger 
and/or the red pencil, (which is exchanged at various points). Often B or G would hold a 
place fixed with a finger, while looking outside, then returning to the map to indicate a 
correspondence. Sometimes B picks up a photo to examine more closely, allowing it to 
be better illuminated by the light coming in his left window. 



Figure 4. At time 18:06 G says, “You’re looking down this valley” (on right, 
gesturing with finger and pencil; B on left). (Photo: W. Clancey/NASA Haughton 

Mars Project) 

The last long, 11 minute navigation discussion at 19:00 (Table 5) reflects the 
group’s concern with the topography ahead and uncertainty of what route to take. Near 
the start of the discussion B and G axe almost exclusively relating the world and the map, 
with a route mentioned near the end. Figure 5 shows the view and corresponding map 
area of the “thick benches of snow” they use to confirm their location. 
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Figure 5. Photograph of view out humvee window (left); three snow patches in the 
upper center are believed to correspond to those in the aerial photo (right), 
indicated by G as he says, “I would put them, really, they should be up here; this is 
them here.” Photo is annotated with a waypoint (1055) and the initial rough route 
plan. (Photos: W. Clancev/NASA Haughton Mars Project) 


Table 5. Navigation discussion between B and G during 19:01-03 


(you see) going off to the left of this < gesturing on map>, 
is this cgesturing outside> 

G 

MAP=>WORLD 

(do you see that it deepens? Steepens, the walls? cjust 
slightly looks out>) Yeah, I think it does, it's that down 
there creturning from look out right window> on the right 
(does it?) yeah (I can’t see) yeah it does, yeah you can see 
it <pointing with pencil, looking out right window>, it’s deep 
and that it, it's getting shallow. This is the shallow bit. 

B 

MAP=>WORLD 

You see that snow bank? (yup) that thin strip of snow 
creturns to point to map> (yeah) i think this < referring to 
snow bank> may be down here cpointing on map> 

G 

WORLD=>MAP 

(okay what about these, ah, <pointing with left hand out 
window to right> these three, these thick snow benches 
there, in that little tributary? < referring to what's visible 
outside> where is that?) Is it this? cpointing on map> 
Just running down into here. ..Isn't that < referring to 
tributary> down there? < pointing on map> 

B 

WORLD = >MAP 

(okay... and these guys? con map>) these guys are up 
here < referring to outside> (that's that?) yup, correct 

B 

MAP=>WORLD 

-(the-SR-ovH-meaR—t-he-t-hiek-benches-ofenow-right-acrossj- — - 
right across there? < referring to outside> uhm (... is it 
that? cpointing on map>) I would put them, really, they 
should be up here cpointing on map>, this is them here. 

(oh, yeah) 

B 

WORLD=>MAP 

this river here con map>, coming off, I'm pretty sure, 
that's this here, cpointing out window> 

G 

MAP=> WORLD 
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We're sitting on top of here con map>, I think up here. 
(Really, up there?) no, uh, further forward I think (let's see, 
let me see ctakes the maps and examines in the bright left 
light> ... yeah, I agree.) 

G 

WORLD = > MAP 

(But actually I think that these, these thick snow banks are 
those, the thick ones we see, are here epointing on 
map>, G) uh hunh 

B 

WORLD=>MAP 

(cause we're right there) clooks out right window> 

B 

WORLD=>MAP 

(the thick snow banks are these... con map>) 

B 

WORLD = > MAP 

(and this Intersection con map> is that epoints with 
pencil out front window >) down here c looking out right 
window>. (oh, I see what you mean... yeah that 
intersection is that, yeah, yes., you're right, you're right) 

B 

MAP=> WORLD 

so those esnow banks on map> are them cthe thick snow 
banks outside> (okay) 1 

G 

WORLD=>MAP 

so I think what we want to do is carry on down there 
creferring to outside>, we'll be wanting to go in that 
direction < looking out and down> (you want to head down 
there?) right, 

G 

ROUTE IN WORLD 

in fact that point egesturing out front right>, in the 
distance, may even be (our landmark, right?) 

G 

WORLD=>MAP 

if you put that under here, we're actually not that far off 
(no) that's there, that maps onto there dining up the 
maps>, so we're looking out, that way egesturing 
direction on map> 

G 

WORLD -> MAP 

(if we follow this valley con map>) 

B 

ROUTE ON MAP 

and that dome in the distance creferring to what's outside> 
(right) the top of that may very well be (right) our 1055 

G 

WORLD -> MAP 


The identification of the snow banks is important for orienting the map. This 
localization method is difficult because there are many snow banks, and from a distance 
they are flattened and appear similar. The method is also amazing because it depends on 
the permanence and relative persistence of shapes of the snow banks over four decades. 
In fact, the area is generally snow-covered from September through May at least, so it is 
no coincidence that the photos would be taken during the height of summer when the 
most features are visible, which is when this humvee traverse occurred. 

The subsequent exchange is very different, as the discussion shifts from locating 
the humvee to planning a route (Tablel>)7TEis sequence provides basic information about 
how route decisions were made (analyzed in the next section). 


Table 6. Navigation discussion between B and G during 19:04-08 


(so you see, by following this valley egesturing on 
map>, we head straight towards it) 

B 

ROUTE ON MAP 

yeah, and the question is whether we want to just 
uhm ... cut across there creferring to map> 

G 

ROUTE ON MAP 

Do you want to go to this waypoint? (to this 

G 

ROUTE ON MAP 
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point? <pointing on map>) yeah, (well, to some 
degree, do we need to?) no, I think we could follow 
a ridge through here <gesturing on map with 
finger> 

■ 


(alright... cause we need to connect with our path 
here <pointing to thin line on right map>, right?) 
yeah, that's correct (so why don't we even cut 
across... straight that way?) 

B 

ROUTE ON MAP 

well, it's looking pretty steep and horrible 
creferring to outside>, I would tend to go more to 
the left. I would avoid that valley down there 
(yeah, the lower banks have, have seem to be sort 
of less rocky, you know?) 

. G 

ROUTE IN 
WORLD 

yeah, I would go towards, do you see where that 
first snow bank is <pointing out right window> 
(which one?) << open window>> first one to the 
left <pointing> (right, the big thick one?) yeah, 
(okay) 

G 

WORLD 

if we head down there < points outside throughout 
this description >, and then turn up over the topf 
around the snow banks, not straight across but go 
around (yup) and over the top, 

G 

ROUTE IN 
WORLD 

I think if we carry on to the top, we'll connect up 
with here <points on map> (we have to look out for 
this vaiiey here <pointing on map>, right>?) 
correct 

G 

ROUTE ON MAP 

(so since we're here con map>, suggesting we go 
up around those con map>... actually we’re 
beyond this map, let's get rid of this, we're 
completely on this map.... since we're here con 
map>, these are your, oh, no, these are your 
things creferring to snowbanks>, right?) yeah 

B 

WORLD -> MAP 

(so you're suggesting basically, we go down here 
cgesturing direction on map>, up and around 
these guys) yes and connect into here clooking on 
map and then outside> 

B 

ROUTE ON MAP 

if we can find a way down this pile of shart. It 
looks pretty steep.... maybe one of these scouts 
wants to find a route down... I think once we get to 
the bottom there, it will be much easier to just go 
up to the top around., (you know, I suspect that 
fhp ot’hpr qjrip ic; nninn f*n Hp PY^rfiv likrp fhiO Hip 


POIITP TM 

other side of what (i mean) that (one of these 
hills) I mean, we don't have a choice do we? I 
mean, this is no better than this (no, I agree) we're 
better cut the route shorter (...valley route that we 
can take, an alternative, hold on one second) ccc 
can hear wind, I ask for window to go up)>>> 

G 

WORLD 

(clooking out right window> Yeah, that con map> 
is that coutside>, okay, I'm convinced) 

B 

MAP -> WORLD 

we need to get down (right) into there 

G 

ROUTE IN 
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WORLD 

(right and what's behind <outside?> is that <on 
map?>, okay... can even recognize the snow 
patches.) 

B 

WORLD -> MAP 

(So being here <pointing on map> and wanting to 
be there con map>) yup. I think our first objective 
is to get down into that valley and to get behind 
those snow patches there cpointing oh map> 
(right right.. to get there) once we get there... we 
can stop again, well you can stop further, well, we 
might stop before then, at least we want to stop 
there and figure out how to connect up with this 
bit < referring to thin line on map beyond next 
waypoint> (yup, very good) as soon as we do 
that... we’ll be. ..almost getting onto this 
map. ..we're well on our way then (yup) heading 
into... 

B 

ROUTE ON MAP 


At this point (immediately after G’s last remark in Table 6 ), B says “Very good, 
let's GPS this place., let's call it decision point. .. ” which G specifies, .way point B.” 

B then says, “Okay robotic scouts, we need to go across this valley and up the 
other side, and get around to those thick snow benches that we see on our right, that 
series of big ones? And to get up and behind those.” B and the scouts then discuss further 
how they should proceed. One comments that “we’re really doing well.. .we’ll be there by 
midnight.” 

After a seven minute rest break, the distant scout tells the humvee driver, to 
follow the ATV nearby, in front of the humvee. B comments, “Okay, follow <near scout, 
S 1>. Now, this is a very important piece of information. . . incredibly important.” G says, 
“What?” The observer asks, “Do we believe it?” G responds, “Yes, it is, incredibly 
important. The future of this vehicle rests on this correct decision.” 

B tells G, “Call SI.” G radios: “SI, can you hear?” Someone says, “Are we 
following you, SI?” B asks, “Who ami following? SI?” G asks, “SI, are we following 
you?” B stops the engine and say, “I'm going to have a look guys.” The observer and G 
then comment about dinner and the possibility of a breakdown; G says, “I hope this thing 
makes it there and back... we've not really come up with a contingency plan....” The 
observer says. “This is just slightly worse than the section where I said you wouldn't 
drive on anything worse than this.” G says, “I think when the boulders get as large as the 
tracks then you staid to have a problem. ...” 

So it is clear at this point, 20 minutes before the mechanical problem, that the 
terrain is not good, and the scout’s judgment requires confirmation. During the next 15 
minutes tire scouts continue to look for a good route into the valley, while B goes out to 
ex amin e the situation. G and the observer discuss the limitations of robots and other 
options for finding routes. G shows how the aerial photo and terrain correspond (Figure 
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5) and comments, “Now if we were on Mars yon could never tell that’ s a rough piece of 
land. It’s just impossible to know... another terrace there.., Houston, we have a 
problem...” A few minutes later, B returns and G asks him, “What do you reckon?” 
“Should be able to make it.” G says,“ ... go over the top.” B replies, “We're going to go 
other there, where it’s smooth.” Soon after some rocks became caught in the humvee’s 
tracks and the group was stopped for the night. 

NAVIGATION DIALOGUE ANALYSIS 

The navigation discussions were analyzed for patterns in how topics were related 
sequentially and correlations between topics and the speaker. The overall impression one 
has on reading the transcript is that during navigation discussions B, the driver, is 
focusing on the map both to locate the humvee and gleam information about routes, while 
G is focusing on the character of the landscape and finding a route in the world. Can we 
show this quantitatively by analyzing the structure- of the conversations? 

Table 7 shows every transition that occurs and is the basis of subsequent tables and 
figures that represent patterns. For example, the first two segments in Table 4 are 
represented in Table 7 by “GB” in row labeled World->Map and the column labeled 
Map=>World (i.e., the dialogue transitions from G mentioning World=>Map to B 
mentioning Map=> World). The entry “2BG, 3BB, GB, 2GG” indicates that the GB 
relation only occurred once in the dialogs; however, the transition occurs seven other 
times, including three times when B mentioned World=>Map and immediately after 
mentioned Map=>World. 


Table 7. Sequential topic transitions during navigation discussions (Table 2). Cells 
indicate speakers (e.g., B speaking about Map=>World was twice followed by G 
speaking about WorId=>Map; hence 2BG appears in row 1, column 2). Bold font 
indicates a dominant transition as defined in the text. _____ 


TO 

FROM 

MAP => 
WORLD 

WORLD => 
MAP 

ROUTE ON 
MAP 

ROUTE IN 
WORLD 

MAP => 


2BG,2BB 

BG 

2BG 

WORLD 

GB 

GB,2GG 

GG 

GB 

WORLD => 

2BG,3BB 

BG,3BB 

BG,5BB 

BG,BB 

MAP 

GB,2GG 

2GB, 2GG 

2GB, GG 

GB,3GG 

ROUTE ON 

BG 

2BG 

BG 

3BG,BB 

MAP 

3GG 

GB, 2GG 

GB,GG 

GG 

ROUTE IN 


TRR 

RG 


WORLD 

2GB 

3GB, GG 

2GG 

GB,GG 


The bold font in Table 7 designates a dominant turn-taking .transition. For this 
purpose, BG and BB are viewed as a pair, and GB and GG are a pair. For each pair, we 
want to know if one appears much more frequently (e.g., does GB appear much more 
frequently than GG?). As defined here, for a transition (e.g., GB) to be dominant, its 
partner must be missing (e.g., GG is missing in last row, column 2) or the ratio must be at 
least 3 to 1 (e.g., 3BG is dominant over BB in “Route on Map To Route in World”). 
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Table 8 aggregates all 16 possible transitions as percentages (16 cells add to 
100%). The least frequent transition was for anyone to follow a remark about how the 
map related to the world by another remark of that kind (1.4%), as Table 8 showed it 
occurred only once in the 74 topic transitions. In contrast, identifying a world feature in 
the map (World=>Map) was much more frequently followed by the same kind of remark 
(1'0:&%): Table*”8”nhows"US that each: of the four combinations - occurred-f BG,- BB,-- GB> 
GG), but B was somewhat more likely to make two such observations in sequence (BG 
vs. 3BB). The most common transition (12.2%), World=>Map followed by. a remark 
about the route on the map, is logical because attention has shifted to the map at this 
point. 


Table 9 counts how often the possible transitions^ (BG, BB, GB, GG) occur by 
summing across Table 7. The TOTAL row shows that alternation of the speakers was 
fairly even, though G was somewhat more likely to two location or navigation remarks in 
sequence (GG = 22/74). Most obviously, B is focused somewhat more on the aerial photo 
(map), as 71% of his repeated remarks (BB) are World=>Map and 72% of his remarks 
overall (49% + 23%) focus on or into the map. In contrast, 59% of G’s remarks focus on 
the map. B appears to be “thinking in” the map, while G is more concerned (26% vs. 9% 
for B) with a route in the world (his role as navigator). B focuses somewhat more than G 
on identifying world features in the map (World^Map, 49% vs. 36%). B appears more 
concerned with locating the Humvee, while G brings the topic back to the route. 


Table 8. Percentage of transitions that occur from topic category designated in row 
header to the category in the column (e.g., World=>Map followed by Route on Map 
is most common, 12.2%). 


TO 

FROM 

MAP => 
WORLD 

WORLD => 
MAP 

ROUTE ON 
MAP 

ROUTE IN 
WORLD 

MAP => 
WORLD 

1.4% 

9.5% 

2.7% 

4.1% 

WORLD => 
MAP 

10.8% 

10.8% 

12.2% 

8.1% 

ROUTE ON 
MAP 

5.4% 

6.8% 

4.1% 

6.8% 

ROUTE IN 
WORLD 

2.7% 

8.1% 

4.1% 

2.7% 


?? We use the terms topic, segment, and utterance interchangeably to refer to one of the 
classified sections of the transcript, i.e., a row in Tables 4, 5, or 6. A “transition” is a 
sequence of two topics and a speaker pair, e.g., “Route on Map -> World=>Map” 
corresponds to the last two rows of Table 5 and is designated BG. 


22 



Clancey et al.: Collaborative navigation while exploring unfamiliar terrain 

Table 9. For a given starting topic, percentage (and number) of transitions of each 
type (e.g., when B spoke about WorId=>Map, G spoke next 5 times [BG], and B 
spoke next 12 times [BB]). Total columns indicate how often each person mentioned 
a topic (e.g., B mentioned World=>Map 17 times - 49% of his 35 utterances) and 
sum to 100%. Last column totals the 74 classified segments in the transcripts (i.e., 74 
topics) with the totals shown (e.g. s World=>Map occurs 31 times). In each row, 


G-FBB = GB+GG = 10C 

) %7 " * 







, ■ - 


SPEAKER SEQUENCE 

TOTAL 


FROM TOPIC 

BG 

BB 

GB 

GG 

B 

G 

TOTAL 

MAP => WORLD 

71% 

29% 

50% 

50% 

20% 

15% 

18% 


(5) 

(2) 

(3) 

(3) 

(7) 

(6) 

(13) 

WORLD => MAP j 

29% 

71% 

43% 

57% 

49% 

36% 

42% 



(5) 

(12) 

(6) 

(8) 

(17) 

(14) 

(31) 

ROUTE ON MAP 

88% 

13% 

22% 

78% 

23% 

23% 

'23% 


( 7 ) 

(1) 

(2) 

( 7 ) 

(8) 

(9) 

(17) 

ROUTE IN WORLD 

33% 

67% 

60% 

40% 

9% 

26% 

18% 


(1) 

m 

(6) 

(4) 

(3) 

(10) 

(13) 

TOTAL 

18 

17 

17 

22 

35 

39 

100% 

(74) 


To highlight dominant transitions, percentages in Table 9 are in bold font when 
the speaker’s frequency of mentioning that topic is >=20% (e.g., B utters Map=>World 7 
times = 20% of his 35 utterances) and the frequency for the sequence is >=60%, Thus, 
BG is highlighted for Map=> World, showing that in this event (i.e., when B utters 
Map=>World), G is much more likely to speak next. Similarly, when B utters 
World=>Map, he is much more likely to speak again. In contrast, when either B or G 
mention “Route on Map,” G speaks next. Finally, when G mentions “Route in World,” B 
speaks next. 

Table 10 aggregates Table 7 transitions according to topic (each row adds to 
100%). For example, Map=> World most often (54%) transitions to (i.e., is followed by) 
World=>Map and relatively infrequently transitions to itself (8%). 

Table 11 shows focus shifts aggregated over all topic segments. For this purpose,- 
the focus of “Route in World” and “Map=>World” is World, and the focus of “Route on 
Map” and “World=>Map” is Map. Thus the transition from Map=> World to Route oh 
Map is classified as “From World to Map.” The table reveals that 65% of the topics were 
focusing in the map (sum of MAP row) and 35% were focusing on the world. Referring 
to Table 9 TOTAL column, we observe that 60% of remarks concerned locating the 
humvee (Worlds Map), while 40% concerned the route. 
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Table 10. Percentage of transitions that occur within a category, combining 
speakers (each row adds to 100%). For example, Map=>World transitions to 
World=>Map with frequency of 54%. 


TO 

FROM 

MAP .=> 
WORLD 

WORLD => 
MAP 

ROUTE ON 
MAP 

ROUTE IN 
WORLD 

MAP => 
-WORLD- 

8% 

54% . 



15%. 

23% 

WORLD => 
MAP 

26% 

26% 

29% 

19% 

ROUTE ON 
MAP 

24% ' 

29% 

18% 

29% 

ROUTE IN 
WORLD 

15% 

46% 

23% 

15% 


Table 11. Frequency of fo cus shifts, independent of speaker (total = 100%) 

TO WORLD MAP 
FROM 


WORLD 

11% 

24% 

MAP 

31% 

34% 


Many high frequency patterns can be found, but to narrow the analysis, we list the 
invariants and missing events (Table 12). 


Table 12. Invariant turn-taking patterns and possible interpretations (cf. Table 7) 
Turn-Taking Pattern Interpretation/Restatement 

When Map=> World and Route in World Focusing on the world, the speakers take 
are mentioned in either order, the speakers turns relating map features and articulating 
alternate turns. the route. 

When Map=> World and Route on Map are G alone moves the topic between a map 
mentioned in either order, G is always the route and identifying a map feature in the 
second speaker. world. 

Only G mentions Route on Map B never follows his map route by another 

sequentially. map route remark. 

— 3-GG-butmot-BBrRoute-0irMap-= B— never— follows— his— map— route— by— a- 

Map=> World world<^map relation. 


2GG but not BB: 

Route On Map - World=>Map 

After B mentions Route on Map, if he 
speaks again (20%) he only mentions 
Route in World. 


B’s remarks about the map route are often 
posed as questions, which G answers by 
identifying map features or continuing 
with the map route. 
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3BG but not GB: Route on Map - Route in 
World 

After B focuses on the world, and the 
world is the next focus, G is always 
speaking. 

2GG but not BB: Route In World - Route 
On Map 

2GB but not BG: Route in World - 
Map=>World 

3 GB but not BG: Route, in World - 
World=>Map 

After B mentions Map=>World or Route 
in World, neither mention that topic next. 
After B mentions Route in World, neither 
mention Map=>World next. 

After B or G focuses in the world, only G 
mentions Route on Map next. 


B never follows G’s map route by a world 
route. 

B never makes two sequential references 
to the world. He instead returns focus to 
the map when he speaks again. 

B never follows his world route by a map 
route. 

G never follows B’s world route by a 
worlds map relation. 


After B focuses in the world, they either 
focus on the map or shift between route 
and feature. 

In bringing focus back to the map, G 
always states the route, not feature 
relations. 


Regarding the overall turn-taking, the overall dialogue has about as many BG 
transitions (18, Table 9) as GB (17). Thus, G is somewhat more likely to speak (40/74 = 
54%) because he speaks more often in sequence (22 vs. 17). This may occur because B is 
asking G questions (see Table 5). 

Also, G is more actively shifting the topic between the map and the world: After G 
focuses on the map and the world is the next focus, G is speaking again (9/1 1 =82%). The 
inverse is not true — -after B focuses on the map and the topic shifts to the world, he is less 
frequently the speaker (5/1 2 = 42%). 


Regarding topic, after any statement focusing in the world (Map=>World or Route in 
World), the next statement is more frequently (18/26= 69%) focused in the map. (But the 
converse is not true: after focusing on the map, the next focus is split, 23 vs. 25). In 
particular, after B focuses in the world, the next focus is almost always on the map (8/10 
= 80%; for G the relation is only 50-50). 

Figure 6 shows the dominant transitions of topic and speaker. A focus in the 

world (left side of diagram, Map=> World or Route in World) most often leads to either 
speaker identifying a world feature on the map (World=>Map, Table 10; also 50% of 
next topic, 13/26 transitions), which is overall the most common utterance (Table 9, 42%) 
and constitutes nearly half of B’s utterances (Table 9, column B, 49%). This is followed 
somewhat most often by Route on Map (Table 10, 29%), uttered by B (Figure 6, caption), 
followed by a return to World=>Map (29%)— and whatever is said, it is most often stated 
by G (Table 9, columns BG + GG = 14/17 = 82%). Notice also that World=>Map is 
repeated more than twice as often as any other topic (10.8% of all transitions, Table 8, 
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diagonal) and most of the time by sequential utterances (Table 9, BB + GG =20/31 - 

66 %). 

To determine the significance of the data, a Davison-Hinkley Chi-Square test was 
performed for each table®®. The significance levels (Table 13) indicate that only the data 
in Table 11 data (topic shifts) are likely to be trite (i.e., not due to chance; significance 
level AcS^s-^ramett-hir&qBvers&fieBna^ -L£- 

hours of data are not sufficient to establish statistical significance of the patterns that 
appear to be meaningful. 

26% 



23% 

Route in World > Route on Map 

(sum 92% 6) 


29% 

Figure 6. Strong correlations in topic transitions. Percentages along arrows are 
from Table 10 (54% of the Map->World remarks transition to World->Map vs. 
46% of the Route in World remarks). (Percents in parentheses designate dominant 
speaker for that transition, from Table 9, E.g., B is dominant in transitions from 
World=>Map to the Route on Map (9 transitions = [2GB + 5BB => 7/9 = 78%] + 
[BG + GG = 12%]). 

Table 13. Statistical analysis of primary data tables. Chi-Square and Significance 
Level are based on Davison-Hinkley method applied to one copy of the data. 
Strength indicates how many times the data would need to be replicated for 

significance level < = .05. • 

Sig 

— — r —Table- — Chi-Square— — Level— —Strength- 

7 64.23 0.43 2 ' 

8 19.19 0.59 2 

9 14.53 0.69 3 

11 7.88 0.001 1 


55 Calculations using Davison-Hinkley method are based on Example 4.22, pp. 177 ff, 
and Algorithm 9.1 (balanced bootstrap) pp. 439 ff and Problem 2, p. 488. 
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However, what if the same data were observed again? Using the “boot-strap 
method,” 17 we calculate the “strength” number (i.e., how many copies of the data, the 
sequence of 74 topic transitions, that would be required to reach a significant level). We 
assume that the analysis should be discounted when this number is five or more. One 
(just the original data) means significant, and two means the data would be significant if 
we observed a second data set like the first. This method indicates that the data in Table 
9, which attempts- to -locate speaker-specific patterns, is more likely to be due to chance. 

The patterns aside, one can consider the logic of the various transitions. The shift 
between Map=>World to World=>Map is a natural back and forth process of looking for 
a map feature in the world and trying to identify a world feature in the map. Locating the 
humvee in the map (World=>Map) is a requirement for establishing a route on the map, 
which then naturally needs to be found in the world (Route in World). Indeed, locating 
the humvee in the map and establishing a route in the world are the only two 
requirements of the navigation task (accounting for all of the navigation stops except 'the 
first orientation, Table 2), with relating map and world features only being a means to 
these ends. Of these, relating the world to the map (what is that?) proceeds from a visible 
feature (e.g., a valley, dome, or snow bank), which serves a kind of clue for locating the 
humvee. In contrast, relating a feature on the map to the world is a kind of academic 
exercise, and it is striking that, except for the waypoints, the features identified at first on 
the topographic map were completely ignored later. Thus 34% of all statements relate 
world to map, more than twice as many as Map=> World. 

Of all the topic relations, the transition from Route in World to Map=>World is 
perhaps least logical, because it implies a shift from talking about the landscape to 
relating a feature in the map to the landscape. This explains why speaker exchange 
occurs when focusing On the world (i.e., while one speaker looks at the world, the other 
uses the opportunity to mention a map feature previously held in mind or perhaps under 
his finger). Neither B nor G mention Route in World and Map=> World in sequence (i.e., 
GG and BB are missing in these two cells in Table 9). The only other sequence of topics 
that G omits is repeating Map=> World. 

Notice that many more issues might have been considered in this analysis, such as 
the proportion of questions to assertions, the exchange of the pencil, the methods of 
establishing a common focus in the world or on the map, and the coordination with the 
scouts. 

EV^EUXTION _ OF'OUTCOME“HYPOTHESES — 

Here we abstract from the data to characterize the navigation dialogues more 
broadly and evaluate the navigation-related hypotheses stated earlier. We begin by 
considering some basic questions about the navigation process: 

• What percentage and amount of time was devoted to locating the humvee? 

60% of the utterances concern the location of the humvee (WorldOMap 
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relations), accounting for roughly 18 min (.6 x 29 min, Table 2) or > 25% of 
the paused time (Table 1). 

• Were the calls to stop to locate the humvee or to choose the route? The last 

six of the nine stops, occurring during the last hour, were to locate the humvee 
(e.g., at 18:39, B says, “G, you need to know exactly where we are... 
sometimes we need to stop for maybe 5 minutes...”), which provided 
infarmatiunxetp^ roUteTtading. ~ ~ 

• How were decisions made about location and route? B and G each called for 
a location stop and map check three times during the last hour. B appears to be 
assisting by locating the humvee in the map (World->Map); G connects the 
projected map route to the world. B’s contributions to the navigation are a 
subset of G’s, fitting G’s role as navigator and B’s driving responsibility. 

8 How do they work? Who initiates what? B worries more about location and G 
worries more about route. Accordingly, each becomes more engaged (more 
likely to speak again), after their primary interest is mentioned. 

• Are they explaining and justifying individual conclusions or solving problems 
together? During the two hours analyzed here, G is able to look at the maps 
and relate them to the world during the hour that B is driving the vehicle, so 
he naturally has more observations and thoughts to convey about the route and 
world-map relationships. B almost always presents location relations and 
routes as questions, thus prompting G (this discourse pattern is called 
mitigation 1 *), the navigator, to make assertions. (For example, when G says, 
“What we need to do is just pick out a route,” B replies, “Does it make us go 
over this high point, or? no?”). 

• Are all of the questions and statements necessary for the task? The dialogue is 
sharply focused on location and navigation, except for the interlude before the 
final stop, outlined earlier, when the uncertain route raised broader issues. 

• Who stated the route first? Of 30 route remarks, B states 1 1 and G states 1 9. 

• Could decisions have been made without stopping? Apparently not. The stops 
indicate that the humvee is moving too fast (< 10 km/hr!) for the navigator to 
follow the location on the map. Thus, if they didn’t stop they would become 
lost. Also, the engine noise made it difficult to talk, and clearly the pilot 
contributes to identifying the location on the map (his primary focus). 

• Does the navigator require help or just prefer to do the job together? The 
navigator needs some kind of assistance, but stopping was due to losing track 
of the current location on the map, not because of needing the pilot’s help. 

• Were the scouts helping? The scouts role was to assist in route finding, but 
' their paths complicated the location problem as they chose paths amenable to 

riding on an ATV. 

At this point, we can directly address the hypotheses stated earlier about navigation- 
related problems that prevented the humvee crew from reaching the coast: 
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_ v. Aerial photographs were insufficient for navigation: Evidently,. true— locating 

the humvee caused significant overhead, including forced stops, and distracted 
the navigator from working with the scouts to plan routes. 

s Route needed to be planned more globally: True— local obstacles for the 
scouts were leading to unplanned detours, which introduced uncertainty in the 
owfali"route" aM“nrahe“ contfnaous repliEniHg' necessary. 45 - min were spent 
finding a local route into a valley, which then caused a mechanical failure. 
Such terrain must be detected and avoided well in advance (preferably before 
the trip leaves base camp). 

• Scouts didn 't provide needed local obstacle and route information: True — the 
crew learned that a rocky hillside affects an A TV and a humvee on tracks 
differently. The ATVs could not move sufficiently quickly to try different 
routes in advance of the humvee; the drivers were also impaired by rain and 
inadequate communications. The navigator was too busy locating the humvee 
to direct the scouts more appropriately. The scouts were not benefiting from 
the aerial photos at all; their lack of shared orientation is evident when they 
come together to talk (Figure 7). 



Figure 7. G (middle) relates his map understanding to the scouts’ conflicting sense 
of direction. (Photo: W. Clancey/NASA Haughton Mars Project) 

CONCLUSIONS-, 

The humvee traverse to the north coast of Devon Island provided an excellent 
authentic, yet experimental setting for studying how people navigate in unfamiliar terrain. 
The data show many intriguing patterns, which suggest that the pilot was interested to. use 
the map (aerial photo) for locating the humvee and finding a route, while the co-pilot’s 
interest reflected his role of finding a route in the world. The study relates to previous 
work that shows the inseparability of gesture and speech in such dialogs, as well as the 
regularity of conversational turn-taking. The patterns revealed here could be applied 
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directly for constructing a computer system that could use other sources of information 
(such as GPS and other maps) to naturally contribute to the human dialogue, in the 
manner of CapCom. More work is required to separate individual preferences in this 
particular crew from more generally shared patterns of narrative and reasoning (e.g., what 
would happen if B and G swapped roles?). 

The eXpeite^rdemonsttafed' clearly fheTmportance ofTiaving sufficient tools to 
locate the vehicle and to plan routes globally, avoiding terrain that will imperil the 
vehicle and its crew or cause long detours. Automatically locating the humvee on a map 
shared by the scouts would allow the navigator to spend more time looking ahead to plan 
routes and get relevant information from the scouts (e.g., by having them test particular 
paths). 

GPS would solve the location problem, but of course could not in itself generate a 
route plan. Some other method must be invented to find a route in advance, with robots or 
scout vehicles, perhaps using satellite or aerial photos to detect problematic terrain. Note 
that as the mechanical problem revealed, the issue is not merely finding a sufficiently 
level path, but identifying surfaces that are difficult or risky for the vehicle being used 
(e.g., consider how Opportunity was stuck on Meridiani Planum for a month in 2005). 
Impassable routes might be color-coded or shaded, just as Mars Exploration Rover 
planners marked out hillside areas where the sun’s angle would prevent the solar cells 
from sufficiently recharging the. batteries. 

The planned route might be superimposed on aerial and/or topographic maps, and 
depicted as 3-d fly-through visualizations. The need to stop periodically to reflect on the 
quality of the route is certainly not precluded, and might be combined with rock and 
mineral sampling, deploying instruments, or resting. The difficulty of relating aerial and 
local perspectives is obvious in the navigation dialogues, and was mentioned by Cernan 
in his journal about Apollo 17: 

When I was near Nansen, I wanted to be sure in my mind what was to the north and 
what was back to the east, even if I couldn't see very far in those directions from the 
ground. I wanted to have a feel for where I was; not from a navigation point of view 
but from a geographic point of view. And then I wanted to understand the 
topography, which you can't fully appreciate until you literally get down to the 
surface. From orbit, Tracy's Rock (at Station 6) doesn't mean anything; but, when you 
get down in there, you can appreciate the slopes and the sizes of the boulders and the 
craters. 

Following the Apollo CapCom model, rather than limiting conversations back to 
base camp by design, why not rely on someone there to help the crew? An aerial balloon 
or high-resolution satellite images might have been used by an assistant back at camp or 
even on Earth, comparing with photos sent back from the humvee to confirm and replan 
routes. 

The difficulty of communicating with the scouts demonstrates that considerable 
thought must be devoted to designing robots that would perform a similar task. First, 
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teleoperation from the vehicle would require at least a third crew member, which is 
plausible. A good follow-up experiment might involve having a third person in the 
humvee mediate the work of the scouts by continuous communication with them on a 
voice loop. The scout interactions shows that their actions must be more restricted to 
certain kinds of actions and follow defined protocols, to provide useful information for 
design of robotic scouts. This in turn requires considering more carefully what kind of 
information" is' desiT'e'd' and' how' it should be conveyed:' For example,” the distant' scout 
might be tasked with only confirming a globally planned route, whose path would be 
visible on a display to the crew and known by the scout (whether a person or robot). 

How might a computer agent have helped the crew? Notice that the. navigator and 
pilot are coordinating the routes of the two scouts, the proximal path of the humvee, and 
the global route of the humvee. Add to this work the need perhaps to periodically manage 
the humvee’ s transport and life-support systems, and to intervene to teleoperate a stuck 
robot (18:46 — “Uh oh, our robot's broken... how you doing <S1>?). Aside from basic 
location tracking and automating reports back to the support crew at base camp, an agent 
might providing basic caution and warning to the vehicle crew about systems and the 
state of the robots. In designing such agents, the present experiment shows the value of 
well-defined protocols, even in such real-life situations, so particular methods can be 
tested. In this respect, the operational test was a success, as the experiment established 
that the navigation method was inadequate for the task: In such terrain with such a 
vehicle, efficient, reliable navigation requires an automated positioning system and 
detailed planning on maps with better than 1 m resolution. 

As we believe the experiment and analysis shows, we are ready to go back to 
Devon Island and try again with different equipment and new procedures. Surely this is 
the advantage of an analog site in preparing for planetary expeditions: The chance to try 
and try again, so when we go to Mars, we will know what to build and how to prepare. 
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